Full Raw Disk Backup

From Kicksecure
Jump to navigation Jump to search

1 to 1 disk backup

Introduction[edit]

TODO: what is a full raw disk backup

It is not possible to backup an operating system installed on an internal disk such as for example Qubes OS while that operating system is running from that operating system.

The advantage of a full raw disk backup is that it should be bootable either by:

  • disconnecting the internal disk and booting from the external backup disk, OR
  • changing the disk ids (undocumented) (of either the internal disk or external disk after the backup), OR
  • using a different computer with compatible hardware to boot from the external disk.

Unfortunately without either above method any attempt to boot the external drive after the backup might likely lead to actually boot the internal drive because these share the same disk ids after the raw disk backup.

Prerequisite Knowledge[edit]

  • How to install an operating system (Debian) on a external USB drive.

Prerequisite Exercises[edit]

echo Output Redirection Essentials[edit]

This should be exercised in a safe environment such as in a disposable VM.

1. Learn the syntax.

Do not run the following command.

  • > - the greater-than sign - means the output file to write to.
  • echo "input content" > path-to-file-name

2. Exercise the redirect to file redirection.

echo "test output" > test-output-file

2. View the created file.

cat test-output-file

3. Sample printout. 

test output

4. Exercise more.

Based on above example, keep changing the file content and output file names until this becomes clear.

PipeViewer Read-Only Essentials[edit]

This should be exercised in a safe environment such as in a disposable VM.

1. Create a new file such as for example:

Open file file-input in a text editor of your choice as a regular, non-root user.

If you are using a graphical environment, run. mousepad file-input

If you are using a terminal, run. nano file-input

2. Add some contents such as for example:

test input

3. Save and close the file:

4. Learn the syntax.

Do not run the following command.

  • < - the less-than sign - means the input file to read from.
  • pv < path-to-file-name-to-read-from

5. Test reading from the file.

pv < file-input

6. Read the output.

Sample output. 

test input
11.0 B 0:00:00 [34.9KiB/s] [================================================================>] 100%

7. Exercise more.

Based on above example, keep changing the input file names until this becomes clear.

PipeViewer Read-Write Essentials[edit]

This should be exercised in a safe environment such as in a disposable VM.

1. Create a new file such as for example:

Open file file-input in a text editor of your choice as a regular, non-root user.

If you are using a graphical environment, run. mousepad file-input

If you are using a terminal, run. nano file-input

2. Add some contents such as for example:

test input

3. Save and close the file.

4. Learn the syntax.

Do not run the following command.

  • < - the less-than sign - means the input file to read from.
  • > - the greater-than sign - means the output file to write to.
  • pv < path-to-file-name-to-read-from > path-to-file-name-to-write-to

5. Test reading from an input file and redirecting the output to another output file.

pv < file-input > file-output

6. Read the output.

Sample output. 

11.0 B 0:00:00 [ 149KiB/s] [================================================================>] 100%

7. Compare the files.

View the two different files. These should look the same.

cat file-input

cat file-output

8. Exercise more.

Based on above example, keep changing the input files and output files until this becomes clear.

Prerequisites[edit]

  • An installed operating system (such as for example Qubes OS) on an (internal) disk.
  • An operating system installed on an external disk, most likely USB such as for example Debian.
  • A separate external backup disk where the backup should be stored.

Information Gathering[edit]

Qubes Users Recommendations[edit]

For educational purposes, it is useful to run gparted and gnome-disks from the Qubes installation which should backed up as full raw disk backup.

Tested in Qubes R4.0 only. In later Qubes versions with untrusted storage domain, these instructions might need changes.

gparted is a disk partitioning tool which will be used as an easy way to find out how many hard drives the system has and what size they have. gnome-disks is a similar utility.

Disconnect any unneeded storage devices.

In dom0, install gparted and gnome-disks.

sudo qubes-dom0-update gparted gnome-disk-utility

In dom0, run gparted.

sudo --set-home gparted

Check the arrow down button below the X (which would close the window) if there are multiple disks.

In dom0, run gnome-disks.

gnome-disks

Make notes. For example.

Note: Modify these notes according to what can be seen in gparted and gnome-disks. 

/dev/sda (476.91 GiB)

Obviously easiest if there is only 1 disk. Assuming there is only 1 disk.

Recommendations[edit]

1. Exercise with completely different test hardware.

It is recommended against to apply this procedure with production hardware if doing this procedure for the first time.

Since data loss is possible if making a mistake during the raw disk backup procedure, it is recommended to exercise the procedure with completely different hardware. Such as a second computer as well as an external boot drive and backup disk that does not contain any important data.

2. Creating a backup date note file.

After booting from the internal disk (which should be backed up).

Create a text file with a small explanation for yourself "Today is day x with date y and time z prior backup number 1."

This will later be handy when doing a restoration test.

3. USB boot operating system should have a graphical diff viewer such as meld installed as well as the following tools.

After booting the operating system from USB.

Install package(s) meld lxqt-sudo gnome-disk-utility gsmartcontrol gparted pv lshw hwinfo ddrescue following these instructions

1 Platform specific notice.

2 Update the package lists and upgrade the system The Web Archive Onion Version .

sudo apt update && sudo apt full-upgrade

3 Install the meld lxqt-sudo gnome-disk-utility gsmartcontrol gparted pv lshw hwinfo ddrescue package(s).

Using apt command line --no-install-recommends option The Web Archive Onion Version is in most cases optional.

sudo apt install --no-install-recommends meld lxqt-sudo gnome-disk-utility gsmartcontrol gparted pv lshw hwinfo ddrescue

4 Platform specific notice.

5 Done.

The procedure of installing package(s) meld lxqt-sudo gnome-disk-utility gsmartcontrol gparted pv lshw hwinfo ddrescue is complete.

Backup High Level Overview[edit]

1. Boot from external USB drive.

2. Find out the device paths of the internal drive and the USB boot drive.

3. Find out the device path of the USB backup drive.

4. Use pv to read from the internal drive and to write to the USB backup drive.

5. Restoration test. (Optional but highly recommended.)

Backup Instructions[edit]

In Linux, unfortunately device names and device paths are non-deterministic, unpredictable, might change with kernel versions and operating system upgrades.

A raw backup with the pv can lead to data loss if used incorrectly as pv is a very powerful tool.

For example, sda is a device name and /dev/sda is a device path. Other device path examples are dev/sdb, dev/sdc.

The actual pv command is not very difficult but the device paths need to be carefully determined before starting the backup, otherwise data loss is at risk.

1. Boot from external USB disk.

2. Do not attach any other disks at this time.

If any other disks are already attached, remove them for now for simplicity.

3. Write output of fdisk to file "old".

sudo fdisk -l > old

4. Information gathering with a few hard drive utilities.

Using lshw.

sudo lshw -class disk

Using hwinfo.

sudo hwinfo --disk

See what disks are currently attached with an alternative tool such as gnome-disks as well.

Try:

gnome-disks

Try gsmartcontrol.

gsmartcontrol-root

See what disks are currently attached with an alternative tool such as gparted as well.

Try:

  • Start gparted from start menu if it can be found there.
  • Try with pkexec if that works: /usr/sbin/gparted
  • Or try with lxsudo if that works: lxsudo /usr/sbin/gparted

4. Attach another disk, the external USB disk which should be used for the backup.

5. Write output of disk to file "new".

sudo fdisk -l > new

6. Compare file "old" with file "new" using "diff".

diff old new

7. Compare file "old" with file "new" using "diff".

meld old new

8. View contents of file /etc/fstab.

cat /etc/fstab

Example printout. 

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
/dev/mapper/debian--vg-root /               ext4    errors=remount-ro 0       1

# /boot was on /dev/sdb1 during installation
UUID=86983f37-38db-401f-889a-bc93d83a3be4 /boot           ext2    defaults        0       2

#/dev/mapper/debian--vg-swap_1 none            swap    sw              0       0

Watch out for the UUID= field. In above example it is 86983f37-38db-401f-889a-bc93d83a3be4.

8. View output of the blkid.

For informational purposes only.

sudo blkid

9. Confirm the device path of the boot device.

Compare output of file /etc/fstab with blkid command.

For example. Note: replace UUID with the actual UUID from /etc/fstab.

sudo blkid | grep 86983f37-38db-401f-889a-bc93d83a3be4

Sample printout. 

/dev/sdb1: UUID="86983f37-38db-401f-889a-bc93d83a3be4" TYPE="ext4" PARTUUID="decf0fe9-01"

In above example, the device path of the boot disk is /dev/sdb. Not /dev/sdb1. The 1 means partition number 1. When making raw disk backups of the full disk, partition numbers must be omitted. Otherwise it would just be a partition backup. In this case, the backup would be unbootable and couldn't be easily restored.

10. Make some notes such as. 

Qubes internal disk 476.91 GiB
Debian external boot disk 931.42 GiB
Backup external disk 931.41 GiB

11. Note the device paths.

Write down the device paths of the internal disk, the external USB boot disk and the external USB backup disk with help according the above instructions.

For the author of this wiki page, /dev/sda was the (Qubes) internal disk, /dev/sdb the (Debian) USB boot disk and /dev/sdc the USB backup drive. This might be different for readers!

12. Note that device paths can change after reboot.

Note that device paths are unfortunately unstable. It happened to the author of this wiki page that /dev/sda and /dev/sdb paths were swapped. It is therefore required to re-identify device paths every time the computer is rebooted.

13. Become root. [1]

sudo su

14. Backup.

Explanation:

  • < - the less-than sign - means the input device to read from
  • > - the greater-than sign - means the output device to write to.

Syntax:

pv < /dev/xxx > /dev/yyy

Note:

  • Replace /dev/xxx with the actual device path of the drive which should be backed up.
  • Replace /dev/yyy with the actual device path of the drive where the backup should be stored.

Examples:

Notice: DATA LOSS POSSIBLE if used incorrectly! Do not use this without prior verification of the device paths!

  • pv < /dev/sda > /dev/sdc

  • pv < /dev/nvme0n1 > /dev/sdc

[2]

The backup might take a long time.

15. Check exit code.

echo $?

Expected output if success. 

0

16. Switch back to normal user.

If the user was previously running sudo su, it should now be undone. Switch back to normal user by running the following command.

exit

17. Done.

Backup is complete.

18. Restoration test.

Without restoration test, it's unclear if the backup could be restored in case needed.

Backup Verification[edit]

verification without progress meter[edit]

size=$(sudo blockdev --getsize64 /dev/nvme0n1)

sudo cmp --bytes=$size /dev/nvme0n1 /dev/sdb

verification with progress meter[edit]

Small script with progress meter. Untested.

original_disk and backup_disk need to be adjusted.

#!/bin/bash set -x set -e set -o pipefail original_disk=/dev/nvme0n1 backup_disk=/dev/sdb test -e "$original_disk" test -e "$backup_disk" ## Get the size of the original disk in bytes. original_size=$(blockdev --getsize64 "$original_disk") ## Using cp with --bytes="$original_size" to ensure that the comparison stops ## at the size of the original disk, even if the backup disk is larger. ## This way, the comparison is not confused if the backup_disk is larger. pv "$original_disk" | cmp --bytes="$original_size" /dev/stdin "$backup_disk" ## Alternative. Untested. #ddrescue --verbose --no-scrape --size="$original_size" "$original_disk" "$backup_disk" ~/logfile

Restoration Test High Level Overview[edit]

Before a restoration test can be performed, a backup is required. Once a backup has been created, attempt the following restoration test instructions.

1. Boot from internal USB drive.

2. Create a (or update) a backup date note file.

Create a text file with a small explanation for yourself "This is prior restoration test. The current date is replace-actual-date. The current time is actual-time. This note should no longer exist after the restoration test."

3. Shut down.

4. Boot from external USB drive.

5. Find out the device paths of the internal drive and the USB boot drive.

6. Find out the device path of the USB backup drive.

7. Use pv to read from the USB backup drive and to write to the internal drive. (Vice versa backup procedure.)

8. Shut down.

9. Boot from internal disk to test if the restoration was successful.

10. Check the backup date note file which was created in step 2.

The note "This is prior restoration test." is now expected to be gone. If the note still exists, then restoration has not been successful. Make sure step 2, the update of the backup date note file has not been previously skipped.

11. Done.

Restoration test has been completed.

Restoration Example[edit]

WARNING: UNTESTED

NOTE: Modify for your needs. See above.

size=$(sudo blockdev --getsize64 /dev/nvme0n1)

sudo su 

pv -pterb -s "$size" /dev/sdb > /dev/nvme0n1

Qubes Specific[edit]

Alternatives[edit]

Potential alternatives, untested by the author of this wiki page.

https://www.tecmint.com/linux-disk-cloning-tools/archive.org

Footnotes[edit]

  1. ↑ Using sudo is unfortunately not possible. At least not without complicating the following pv command. Becoming root is required because shell redirection are a bash feature, not a sudo feature. These are set up before becoming sudo. Alternatively, it would be possible to write a small script, pasting the command there and executing that script using sudo.
  2. ↑
Your support makes
all the difference!

We believe security software like Kicksecure needs to remain Open Source and independent. Would you help sustain and grow the project? Learn more about our 12 year success story and maybe DONATE!

Retrieved from "https://www.kicksecure.com/w/index.php?title=Raw_Disk_Backup&oldid=84572"